1. Field of the Invention
This invention relates to a multi-feedback circuit apparatus, and more particularly to a multi-feedback circuit apparatus for performing advanced parallel computation modeled after a neural network of a living body.
2. Description of the Prior Art
A multi-feedback circuit apparatus which is called the Hopfield model proposed by Dr. J. J. Hopfield, et al. has been known. The apparatus is to be applied to various fields such as character recognition, speech recognition, optimization in general, and neuro-computers. The Hopfield model is one of the techniques useful to model a neural network of a living body by the use of electronic circuits. Specifically, nerve cell circuits, which exhibit nonlinear analog response are interconnected through symmetrical synapse nodes so as to constitute a neural network model. These nerve cell circuits are modeled after nerve cells of a living body (see Science, vol. 233, Aug. 8, 1986). Here, the Hopfield model will be described with reference to FIG. 10.
In FIG. 10, a plurality of operational amplifiers OP1, OP2 . . . OPi, which respectively have sigmoid input-output characteristics, are provided in parallel. The respective positive input terminals (+) of the operational amplifiers OP1, OP2 . . . OPi are grounded. Further, input impedance such as Z1, Z2 . . . Zi, a plurality of synapse nodes, and input current sources I1, I2 . . . Ii are connected to the respective negative input terminals (-) of the operational amplifiers OP1, OP2 . . . OPi. Here, the synapse nodes are constituted by feedback resistors R11 . . . R1i, R1j, R21 . . . R2i, R2j, and Ri1 . . . Rii, Rij. The input impedance such as Z1, Z2 . . . Zi are circuits having input resistors .rho.1, .rho.2 . . . .rho.i and input capacitors C1, C2 . . . Ci, connected in parallel, respectively.
The output terminals OUT1, OUT2 . . . OUTi of the operational amplifiers OP1, OP2 . . . OPi are connected to the feedback resistors R11 . . . R1i, R1j, R21 . . . R2i, R2j, and Ri1 . . . Rii, Rij through feedback paths F1. . . Fi, Fj. As a result, a synapse node square matrix configuration can be formed. The conductances of the respective feedback resistors are so designed as to be symmetrical with respect to a dot-and-dash diagonal line. In other words, the feedback resistors R11 and Rij, R21 and Rii, and R1i and R2j are respectively determined to be the identiCal conductance. Hereinafter, such conductance is typically represented by Tij.
The local behavior of this model can be expressed by the following ordinary differential equation; ##EQU1## where Ui represents input currents of the amplifiers OP1, OP2 . . . OPi, and t represents time. Further, g has the following characteristics; EQU g : (-.infin., +.infin.) (-1, +1),
in the presence of an odd function, a monotonous increment and an inverse function which are primarily differentiable. However, a sigmoid function is usually selected as g.
The overall behavior of this model circuit will be described. The response of the circuit indicates that the behavior of the circuit is to be converged on a stable equilibrium solution. In order to mathematically secure this, a square matrix which has the above-described Tij as elements must satisfy prescribed conditions such as being symmetrical and the like. This concept has been explained by taking the following energy function, which is based on the variation principle, into consideration; ##EQU2##
Namely, the overall behavior of the Hopfield model circuit is such that the Outputs V1, V2 . . . Vi of the amplifiers OP1, OP2 . . . OPi are varied so as to decrease the value of this energy E. In general, a stable equilibrium solution of this circuit can be obtained using. Tij as parameters. Here, assume that "NP complete" (non-polynominal order compliexity) problems, which are called combination problems and cannot be solved by the order of a polynominal expression, are translated into these parameters. In this case, this circuit solves the given problems as optimization problems (see U.S. Pat. No. 4,719,591).
However, when the Hopfield model is simulated by numerical calculation or realized by actual electronic circuits, the following undesired phenomena are likely to arise. Specifically, they are;
(1) The apparatus inevitably finds an equilibrium solution at a pseudo-minimum value before the energy E reaches its true minimum value.
(2) The apparatus oscillates or exhibits so-called chaos phenomena, in which the apparatus inevitably follows the locus of oscillation. The phenomenon (1) is caused by the fact that an equilibrium solution, in which the energy E reaches its true minimum value, is invariably equal to a sOlutiOn in the case when t is assumed as t .fwdarw..infin.. However, the reverse does not necessarily hold true. In this case, a solution to be obtained does not necessarily satisfy limitations included in a given problem. Thus, some considerations are generally needed to escape from an undesired minimum solution.
Conventionally, there has been a simulated annealing technique to meet such need by use of electronic circuits (see Science, vol 220, May 13, 1983). This technique was verified by Dr. Geman and Geman (see IEEE Trans., vol PAMI-6, 1984). According to the technique, a strategic temperature control can be realized so that the energy E reaches its true minimum value at a time when the circuit finds an equilibrium solution. In Other words, the undesired phenomenon (1) can be eliminated by use of electronic circuits. For example, noises having prescribed probability distribution are superimposed on input signals of the amplifiers (see Neural Information processing System, Denver, 1987). The oscillation phenomena (2) can occur when the square matrix that includes the sYnapse nodes is not symmetrical. In general, when some of the synapse nodes are missing at random in the Hopfield model circuit, the dynamic stability of the model circuit is substantially free from adverse effects. However, when synapse nodes are asymmetric in terms of the structure of the Hopfield model circuit, the oscillation phenomena (2) inevitably occurs therein.
In some particular cases, the harmonized oscillation derived from the oscillating phenomena and the oscillatiOn phenomena might become a required calculation result. For example, in the case of a pattern generator in the central nervous system of a measuring worm, such oscillating phenomena are necessary to achieve its peristaltic motion.
However, in most cases, such oscillating phenomena become very disadvantageous when the stable equilibrium solution at a minimum energy value is assumed to be the calculation results of the circuit ParticularlY when the circuit is to be realized be use of an integrated circuit, the variations in the values of respective elements, which occur inevitably in the manufacturing process, cause the synapse nodes in the circuit to be asymmetric. Specifically, the feedback resistors R11 and Rij, R21 and Rij, and R1i and R2j, the respective pairs of which must have the same conductance, inevitably have a different conductance. As a result, the above-described disadvantages (1) and (2) cannot be avoided.